Infinite horizon value functions in the Wasserstein spaces
نویسندگان
چکیده
We perform a systematic study of optimization problems in the Wasserstein spaces that are analogs of infinite horizon, deterministic control problems. We derive necessary conditions on action minimizing paths and present a sufficient condition for their existence. We also verify that the corresponding generalized value functions are a type of viscosity solution of a time independent, Hamilton–Jacobi equation in the space of probability measures. Finally, we prove a special case of a conjecture involving the subdifferential of generalized value functions and their relation to action minimizing paths. © 2014 Elsevier Inc. All rights reserved.
منابع مشابه
Convergence of trajectories in infinite horizon optimization
In this paper, we investigate the convergence of a sequence of minimizing trajectories in infinite horizon optimization problems. The convergence is considered in the sense of ideals and their particular case called the statistical convergence. The optimality is defined as a total cost over the infinite horizon.
متن کاملConvergence Proofs of Least Squares Policy Iteration Algorithm for High-Dimensional Infinite Horizon Markov Decision Process Problems
Most of the current theory for dynamic programming algorithms focuses on finite state, finite action Markov decision problems, with a paucity of theory for the convergence of approximation algorithms with continuous states. In this paper we propose a policy iteration algorithm for infinite-horizon Markov decision problems where the state and action spaces are continuous and the expectation cann...
متن کاملGradient Flows on Wasserstein Spaces over Compact Alexandrov Spaces
We establish the existence of Euclidean tangent cones on Wasserstein spaces over compact Alexandrov spaces of curvature bounded below. By using this Riemannian structure, we formulate and construct gradient flows of functions on such spaces. If the underlying space is a Riemannian manifold of nonnegative sectional curvature, then our gradient flow of the free energy produces a solution of the l...
متن کاملSolving infinite horizon optimal control problems of nonlinear interconnected large-scale dynamic systems via a Haar wavelet collocation scheme
We consider an approximation scheme using Haar wavelets for solving a class of infinite horizon optimal control problems (OCP's) of nonlinear interconnected large-scale dynamic systems. A computational method based on Haar wavelets in the time-domain is proposed for solving the optimal control problem. Haar wavelets integral operational matrix and direct collocation method are utilized to find ...
متن کاملA Version of the Euler Equation in Discounted Markov Decision Processes
This paper deals with Markov decision processes MDPs on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique DP . Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal ...
متن کامل